Publish/subscribe (pub/sub) has become a popular communication paradigm that provides a loosely coupled form of interaction among many publishing data sources and many subscribing data sinks. One type of such system is topic-based pub/sub, wherein publishers associate each publication message with one or more specific topics, and subscribers register their interests in a subset of all topics. In many pub/sub systems clients interact with a single server, also referred to as “broker”, that is responsible for providing the required pub/sub services, such as delivering a message published on a topic to clients that subscribed to this topic. In order to provide a large scale pub/sub service, multiple brokers are grouped together to form a cluster. The brokers in the cluster collaborate and exchange information in order to provide pub/sub service to a large group of clients.
Some Internet of Things (IoT) systems rely on the Message Queue Telemetry Transport (MQTT) protocol or other pub/sub protocols to offer advanced connectivity and communication between enterprise applications and IoT devices. A cloud-hosted IoT infrastructure may be expected to support tens or hundreds of millions of IoT devices. To accommodate such large-scale workloads, deployment of many MQTT brokers on geographically distributed data centers may be desired or required.
MQTT is a pub/sub messaging protocol, designed for low resources devices and networks. MQTT treats topics as a hierarchy, using slash (/) as a separator. When a client publishes a message, the publication is for a specific topic. Clients can receive published messages by subscribing to the topics they are interested in. A subscription could be for a specific, exact topic (e.g., “/weather/usa/new-york”), or it could be a wildcard subscription for multiple topics (e.g., “/weather/usa/#”). MQTT defines two wildcards: ‘+’ and ‘#’. The symbol + represents a single level of hierarchy in the subscription, and the symbol # stands for zero or more levels of hierarchy. Specifically, in MQTT, # must be the last symbol in a wildcard subscription. However, other pub/sub protocols may allow for wildcard subscriptions with # appearing at any level (e.g., “/#/usa/new-york”).
One feature defined by MQTT is support for retained messages. The publisher may set any message to be retained for a topic. The broker will keep the retained message even after sending it to all current subscribers. If a client issues a new subscription that matches the topic of the retained message, then the client will receive the message. Under MQTT, at most one retained message is kept per topic; that is, if a topic has a retained message and a new retained message is received then the new message will replace the previous message and the old message will be discarded. However, other protocols may support multiple retained messages per topic. In the context of IoT, retained messages may be used by the connected devices to record their current status for quick update or recovery purposes, for example.